Image processing apparatus, image processing method, and computer readable medium

ABSTRACT

An image processing apparatus includes the following elements. A receiving device receives an image. An extracting device extracts regions from the image received by the receiving device. A selecting device selects a region from among the regions extracted by the extracting device in accordance with a predetermined rule. A measuring device measures luminance values of pixels contained in the region selected by the selecting device. An estimating device estimates a function representing a degree of fog in the image received by the receiving device from the luminance values of the pixels measured by the measuring device. An eliminating device eliminates fog from the image received by the receiving device on the basis of the function estimated by the estimating device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2011-115011 filed May 23, 2011.

BACKGROUND Technical Field

The present invention relates to image processing apparatuses, imageprocessing methods, and computer readable media.

SUMMARY

According to an aspect of the invention, there is provided an imageprocessing apparatus including: a receiving device that receives animage; an extracting device that extracts regions from the imagereceived by the receiving device; a selecting device that selects aregion from among the regions extracted by the extracting device inaccordance with a predetermined rule; a measuring device that measuresluminance values of pixels contained in the region selected by theselecting device; an estimating device that estimates a functionrepresenting a degree of fog in the image received by the receivingdevice from the luminance values of the pixels measured by the measuringdevice; and an eliminating device that eliminates fog from the imagereceived by the receiving device on the basis of the function estimatedby the estimating device.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiment(s) of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a block diagram illustrating conceptual modules forming animage processing apparatus according to an exemplary embodiment;

FIG. 2 is a flowchart illustrating an example of processing performed bythe exemplary embodiment;

FIG. 3 illustrates an example of processing performed by an imagereceiving module;

FIGS. 4 and 5 illustrate an example of processing performed by anon-edge region extracting module;

FIG. 6 illustrates an example of processing performed by aninspection-region selecting module;

FIGS. 7A through 8B illustrate an example of processing performed by apixel-value measuring module;

FIGS. 9A through 10C illustrate an example of processing performed by apixel-value distribution estimating module;

FIGS. 11 through 13 illustrate an example of processing performed by afog eliminating module;

FIGS. 14A through 14F illustrate an example of a technology, whichserves as a basis for the exemplary embodiment; and

FIG. 15 is a block diagram illustrating an example of the hardwareconfiguration of a computer implementing the exemplary embodiment.

DETAILED DESCRIPTION

In order to facilitate the understanding of an exemplary embodiment, atechnology, which serves as a basis for the exemplary embodiment, willfirst be described.

In FIG. 14A, a subject document 1410 to be photographed is a name cardand contains, not only a black and white portion, but also a chromaticcolor portion (e.g., a certain region may be a photograph or may befilled with a solid color different from the color of the paper of thedocument, i.e., a color other than white). More specifically, within thesubject document 1410, the background of a region 1412 is red, thebackground of a region 1414 is white, and a graphic 1416 is a red mark.The subject document 1410 is photographed by using, for example, adigital camera, resulting in a photographed image 1430 shown in FIG.14C. The photographed image 1430 shows that image deterioration, such asa fog image 1420 shown in FIG. 14B, has occurred and is superposed onthe photographed image 1430. There are various reasons why the fog image1420 is generated, such as non-uniformity of illumination, taking apicture facing toward the light, the position of the subject document1410 (i.e., positional relationship among the subject document 1410,illumination, and an image capturing device), and the configuration ofthe lenses of a digital camera (e.g., the center portion of an image isbrighter, while the peripheral portion thereof is darker). FIG. 14Bshows that the fog image 1420 is a gradation image in which the topright is a dark black and the bottom left is white, i.e., the shade ofthe color progressively changes. Accordingly, in the photographed image1430 shown in FIG. 14C, although the top right is a dark black, thebottom left is close to the image of the original subject document 1410.This fog is also referred to as “shadow” or “black fog”.

Fog images may be generated, not only by the use of a digital camera,but also by the use of a scanner. For example, in a so-called flatbedscanner, a pivotable pressure plate is provided on contact glass, andafter setting a document on the contact glass, the pressure plate isclosed and the document is scanned. Documents to be scanned include notonly sheet-like documents, but also book-like documents (books,booklets, etc.). Such a book-like document is also set on the contactglass and is scanned. However, the binding portion of such a document iselevated above the contact glass, and is separated from the focal plane.As a result, image deterioration called “fog” occurs in the resultingscanned image corresponding to a position at which the binding portionof the document has been set.

A description will now be given of an exemplary embodiment of theinvention with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating conceptual modules forming animage processing apparatus according to an exemplary embodiment.

Generally, modules are software (computer programs) components orhardware components that can be logically separated from one another.Accordingly, the modules of this exemplary embodiment are not onlymodules of a computer program, but also modules of a hardwareconfiguration. Accordingly, this exemplary embodiment will also bedescribed in the form of a computer program for allowing a computer tofunction as those modules (a program for causing a computer to executeprogram steps, a program for allowing a computer to function ascorresponding units, a computer program for allowing a computer toimplement corresponding functions), a system, and a method. Whileexpressions such as “store”, “storing”, “being stored”, and equivalentsthereof are used for the sake of description, such expressions indicate,when this exemplary embodiment relates to a computer program, storingthe computer program in a storage device or performing control so thatthe computer program is stored in a storage device. Modules maycorrespond to functions based on a one-on-one relationship. In terms ofimplementation, however, one module may be constituted of one program,or plural modules may be constituted of one program. Conversely, onemodule may be constituted of plural programs. Additionally, pluralmodules may be executed by using a single computer, or one module may beexecuted by using plural computers in a distributed or parallelenvironment. One module may integrate another module therein.Hereinafter, the term “connection” includes, not only physicalconnection, but also logical connection (sending and receiving of data,giving instructions, reference relationship among data elements, etc.).The term “predetermined” means being determined before a certainoperation, and includes the meaning of being determined prior to acertain operation before starting processing of this exemplaryembodiment, and also includes the meaning of being determined prior to acertain operation even after starting processing of this exemplaryembodiment, in accordance with the current situation/state or inaccordance with the previous situation/state.

A system or an apparatus may be realized by connecting plural computers,hardware units, devices, etc., one another via a communication medium,such as a network (including communication based on a one-on-onecorrespondence), or may be realized by a single computer, hardware unit,device, etc. The terms “apparatus” and “system” are used synonymously.The term “system” does not include merely a man-made social “mechanism”(social system).

Additionally, before performing an operation by using a correspondingmodule or when performing plural operations by using a correspondingmodule, target information is read from a storage device, and afterperforming the operation, a processed result is written into the storagedevice. Accordingly, a description of reading from the storage devicebefore an operation or writing into the storage device after anoperation may be omitted. Examples of the storage device may include ahard disk, a random access memory (RAM), an external storage medium, astorage device using a communication circuit, a register in a centralprocessing unit (CPU), etc.

An image processing apparatus shown in FIG. 1 of this exemplaryembodiment serves the function to eliminate fog from images. The imageprocessing apparatus shown in FIG. 1 by way of example includes an imagereceiving module 110, a non-edge region extracting module 120, aninspection-region selecting module 130, a pixel-value measuring module140, a pixel-value distribution estimating module 150, a fog eliminatingmodule 160, and an output module 170.

The image receiving module 110 is connected to the non-edge regionextracting module 120. The image receiving module 110 receives an imageand delivers the received image to the non-edge region extracting module120. Receiving an image includes reading an image by using, for example,a scanner and a camera, receiving an image from an external device byusing, for example, a fax machine, via a communication circuit, readingan image stored in a hard disk (which may be built in a computer or maybe connected to the computer via a network), etc. The image is amulti-valued (including a color image) image. Only one image may bereceived or plural images may be received. The contents of imagesinclude business documents (e.g., the above-described name cards),advertising pamphlets, images drawn on a whiteboard, etc. The imagereceiving module 110 may make correction for the received image byperforming digital filer processing, etc., such correction includingcamera shake correction. An example of processing performed by the imagereceiving module 110 will be discussed later with reference to FIG. 3.

The non-edge region extracting module 120 is connected to the imagereceiving module 110 and the inspection-region selecting module 130. Thenon-edge region extracting module 120 extracts regions from an imagereceived by the image receiving module 110 and delivers the extractedregions to the inspection-region selecting module 130.

The non-edge region extracting module 120 also extracts regions equal toor larger than a predetermined size or regions larger than thepredetermined size, and if the non-edge region extracting module 120 hasnot found such regions, operations of the inspection-region selectingmodule 130, the pixel-value measuring module 140, the pixel-valuedistribution estimating module 150, the fog eliminating module 160, andthe output module 170 may not be performed. In this case, the non-edgeregion extracting module 120 may output information indicating that theabove-described regions have not been found (i.e., elimination of foghas not been performed).

The regions to be extracted are regions in which color values arechanged smoothly and continuously, in other words, edgeless regions orregions surrounded by edges.

An example of the processing performed by the non-edge region extractingmodule 120 will be discussed later with reference to FIGS. 4 and 5.

The inspection-region selecting module 130 is connected to the non-edgeregion extracting module 120 and the pixel-value measuring module 140.The inspection-region selecting module 130 selects a region from theregions extracted by the non-edge region extracting module 120 inaccordance with a predetermined rule, and delivers the selected regionto the pixel-value measuring module 140.

The predetermined rule employed in the inspection-region selectingmodule 130 may be determined by using the sizes of the regions or theluminance or chroma values of the regions extracted by the non-edgeregion extracting module 120. Alternatively, the predetermined rule maybe determined by at least one of the variance of the chroma values ofthe pixels in the extracted regions, the positions of the regions in theimage, and the sizes of the outer peripheries of the regions.

An example of the processing performed by the inspection-regionselecting module 130 will be discussed later with reference to FIG. 6.

The pixel-value measuring module 140 is connected to theinspection-region selecting module 130 and the pixel-value distributionestimating module 150. The pixel-value measuring module 140 measures theluminance values of the pixels in the region selected by theinspection-region selecting module 130, and delivers the measurementresult to the pixel-value distribution estimating module 150. Theluminance values of the pixels are measured by scanning the region in apredetermined direction.

The scanning direction in which the region is scanned for measuring theluminance values of the pixels may be at least one of a horizontaldirection, a vertical direction, an oblique direction, a direction inwhich the region is scanned in an elliptical shape. The scanningdirection may be two directions, such as a combination of the horizontaldirection and the vertical direction, a top right oblique direction anda bottom right oblique direction, or may be one direction, such asscanning the region in an elliptical shape, or may be a combination ofthree scanning directions.

An example of the processing performed by the pixel-value measuringmodule 140 will be discussed later with reference to FIGS. 7A through8B.

The pixel-value distribution estimating module 150 is connected to thepixel-value measuring module 140 and the fog eliminating module 160. Thepixel-value distribution estimating module 150 estimates a functionrepresenting the degree of fog in the image received by the imagereceiving module 110 from the pixel values measured by the pixel-valuemeasuring module 140, and delivers the estimated function to the fogeliminating module 160. The function representing the degree of fog maybe a linear function or a multidimensional function, such as a quadraticfunction.

The pixel-value distribution estimating module 150 may estimate afunction generated by connecting the luminance values of the pixels inthe region selected by the inspection-region selecting module 130.

An example of the processing performed by the pixel-value distributionestimating module 150 will be discussed later with reference to FIGS. 9Athrough 10C.

The fog eliminating module 160 is connected to the pixel-valuedistribution estimating module 150 and the output module 170. The fogeliminating module 160 eliminates fog from the image received by theimage receiving module 110 on the basis of the function estimated by thepixel-value distribution estimating module 150, and delivers the imagefrom which fog has been eliminated to the output module 170.

If the pixel-value measuring module 140 has measured the luminancevalues of the pixels by scanning the region of the image in pluralscanning directions, the fog eliminating module 160 may calculate thevalue of fog at a position in which the scanning directions intersectwith each other, on the basis of the values obtained from pluralfunctions estimated by the pixel-value distribution estimating module150. Concerning pixels which are not positioned on the scanning lines,the values of fog at the positions of such pixels may be calculated byusing the values of fog at the positions of the pixels on the scanninglines in accordance with the distances between the pixels on thescanning lines and the corresponding pixels which are not on thescanning lines. The values of fog indicate the differences between thepixel values of the received image and the pixel values of the originalimage (pixel values of an image which has been photographed without theoccurrence of fog).

An example of the processing performed by the fog eliminating module 160will be discussed later with reference to FIGS. 11 through 13.

The output module 170 is connected to the fog eliminating module 160.The output module 170 receives the image from which fog has beeneliminated by the fog eliminating module 160 and outputs the image.Outputting an image includes printing an image by using, for example, aprinter, displaying an image on, for example, a display, transmitting animage by using an image transmitting apparatus, such as a fax machine,writing an image into an image storage device, such as an imagedatabase, storing an image in a storage medium, such as a memory card,delivering an image to another information processing apparatus, etc.Such an image may be output in association with the image received bythe image receiving module 110.

FIG. 2 is a flowchart illustrating an example of the processingperformed by this exemplary embodiment.

In step S202, the image receiving module 110 receives a subject image.

FIG. 3 illustrates an example of the processing performed by the imagereceiving module 110. A subject document 320 to be photographed isequivalent to the subject document 1410 shown in FIG. 14A. The subjectdocument 320 includes a region having a red background at the left side,a region having a white background at the right side, and a regionrepresenting a red mark at the top right. When the subject document 320is photographed by using an image capturing device 310, fog occurs andis reflected in a resulting image, such as that in a photographed image330. For example, after exchanging name cards, the subject document 320representing one of the name cards is placed on a desk and isphotographed with the image capturing device 310. The image capturingdevice 310 may be an independent digital camera, a digital camera builtin a personal digital assistant e.g., in a cellular phone including asmartphone, or a scanner, etc.

In step S204, the non-edge region extracting module 120 extracts regionsin which color values are changed smoothly and continuously.

FIGS. 4 and 5 illustrate an example of processing performed by thenon-edge region extracting module 120. The non-edge region extractingmodule 120 includes, for example, a sobel filter module 410, abinarizing module 420, a black-and-white reversing module 430, alabeling module 440, and a small-size eliminating module 450.

The sobel filter module 410 is connected to the binarizing module 420,and performs sobel filter processing on the photographed image 330received in step S202 and delivers the resulting image to the binarizingmodule 420. The sobel filter processing is to detect edges of verticallines or horizontal lines by using a filter. In the sobel filterprocessing, lines other than vertical lines and horizontal lines may bedetected depending on the design of the filter.

The binarizing module 420 is connected to the sobel filter module 410and the black-and-white reversing module 430. Upon receiving an image inwhich edges have been detected by the sobel filter module 410, thebinarizing module 420 performs binarizing processing on the image anddelivers the binarized image to the black-and-white reversing module430. As this binarizing processing, a known binarizing processingtechnique may be applied. As a result of this binarizing processing, theedges have turned out to be black and the other portions have turned outto be white.

The black-and-white reversing module 430 is connected to the binarizingmodule 420 and the labeling module 440. Upon receiving the binarizedimage from the binarizing module 420, the black-and-white reversingmodule 430 performs black-and-white reverse processing on the binarizedimage and delivers the resulting image to the labeling module 440. As aresult of this black-and-white reverse processing, the edges are changedto white, and the other portions are changed to black.

The labeling module 440 is connected to the black-and-white reversingmodule 430 and the small-size eliminating module 450. Upon receiving animage subjected to black and white reverse processing by theblack-and-white reversing module 430, the labeling module 440 performslabeling processing on the image and delivers the resulting image to thesmall-size eliminating module 450. As a result of this labelingprocessing, the same label is attached to a black region exhibitingcontinuous black pixels. Accordingly, by extracting the regions attachedwith the same label, regions without an edge can be extracted.

The small-size eliminating module 450 is connected to the labelingmodule 440. Upon receiving an image labeled by the labeling module 440,the small-size eliminating module 450 eliminates noise, whichcorresponds to regions equal to or smaller than a predetermined size orregions smaller than the predetermined size. In other words, as a resultof this processing, regions equal to or greater than the predeterminedsize or regions greater than the predetermined size can be extracted.The area of a region attached with the same label may be determined bycounting the number of pixels attached with the same label or bycalculating the area of a circumscribed rectangle of that region.

An example of the image obtained by performing the above-describedprocessing on the photographed image 330 shown in FIG. 4 is shown inFIG. 5. A region image 510 is a region having a red background at theleft side of the photographed image 330. A region image 520 is a regionhaving a white background at the right side of the photographed image330. A region image 530 is a region having a red mark at the top rightof the photographed image 330. However, those images are binary images,and serve as mask images. More specifically, a logical AND operationbetween the photographed image 330 and the mask images is performed,thereby making it possible to extract the regions corresponding to themask images from the photographed image 330.

The non-edge region extracting module 120 may extract non-edge regionsby performing processing other than the processing using the moduleconfiguration shown in FIG. 4. For example, the non-edge regionextracting module 120 may extract non-edge regions by integrating thesame color pixels. The same color pixels are not restricted to exactlythe same color pixels, but may be color pixels having a predeterminedrelationship. More specifically, a certain pixel may be selected from animage, and pixels which are in contact with the selected pixel and whichhave a predetermined relationship with the selected pixel (e.g., thedistance between the two pixels is equal to or smaller than apredetermined value or the distance between the two pixels is smallerthan the predetermined value in a color space) may be integrated. Then,the above-described integration processing may be similarly performed onthe integrated pixel. As a result, a non-edge region is extracted.

Referring back to the flowchart of FIG. 2, in step S206, theinspection-region selecting module 130 extracts a region suitable forestimating the gradient of the luminance values of fog.

The inspection-region selecting module 130 extracts the characteristicvalues of the regions extracted in step S204. Then, theinspection-region selecting module 130 calculates the value according toequation (1) (discriminant function) for each region, and selects aregion (in this example, the above-described mask image) in accordancewith the calculated values. For example, among the regions extracted instep S204, the region having the maximum value is selected.

$\begin{matrix}{{f\left( {X_{i};w} \right)} = {\sum\limits_{j}\;{w_{j}x_{ij}}}} & (1)\end{matrix}$In equation (1), w_(j) in the right side is a weight, and may be apositive number, 0, or a negative number. If w_(j) is 0, it means thatthe corresponding characteristic value (weight) is not used. If w_(j) isa negative number, the characteristic value acts in the negativedirection. The weight w_(j) is a predetermined value which may be set byan operator or which may be a value obtained by conducting learning byusing supervisor data in which a correct answer is determined amongseveral choices.

In equation (1), x_(ij) in the right side denotes characteristic values.The inspection-region selecting module 130 extracts those characteristicvalues from each region (i). The characteristic values x_(ij) include:

x_(i0) is the width or the height of a region, or may be a width×height(i.e., the area of a circumscribed rectangle);

x_(i1) is the area of a region;

x_(i2) is the pixel density of a region;

x_(i3) is the average of the luminance values (e.g., the value of L* inthe case of an L*ab color space) of the pixels within a region;

x_(i4) is the average of the chroma values (e.g., the values of a and bin the case of an L*ab color space) of the pixels within a region;

x_(i5) is the variance of the chroma values of the pixels within aregion;

x_(i6) is the distance from the center of gravity of a region (maskimage) to the center of the image received in step S202; and

x_(i7) is the length of the peripheral contour of a region/the boundarylength of a circumscribed rectangle of the region.

The above-described characteristic values are merely examples, and othercharacteristic values may be used. Additionally, as one of thecharacteristic values, the average of the luminance or chroma values isused for describing the typical luminance or chroma of the pixels in aregion. However, instead of using the average, a value representing thetypical luminance or chroma of a region may be used, for example, amodal value or a median value may be used.

It is not necessary to use all the characteristic values, and some ofthem may be selected. For example, a discriminant function using one ofthe characteristic values x_(i0), x_(i1), and x_(i2) may be employed.This is because a large area is suitable for estimating a functionrepresenting the degree of fog.

Further, x_(i3) or x_(i4) may be added in order to select a backgroundregion of the document. This is because a background region is alsosuitable for estimating a function indicating the degree of fog. Thebackground region is generally white, and thus, it has a high luminancevalue and a low chroma value. The weight w₄ of the chroma value x_(i4)is a negative value.

The above-described characteristic values are sufficient to estimate thedegree of fog superposed on images, such as name cards, documents, andwhiteboards having a white background. However, such characteristicvalues may not be sufficient for images obtained by photographing adocument, such as a name card, in which a landscape picture occupies thehalf of the area of the document (the area of the white background isrelatively narrow), or for images obtained by photographing a whiteboardfacing toward the light (the surrounding portion of the photographedimage has a high luminance value).

Accordingly, x_(i5) through x_(i7) may be added.

Generally, the variance of chroma values of a landscape picture ishigher than that of a background. Thus, x_(i5) may be used so that aregion having a smaller variance of chroma values can be selected whilepreventing a landscape picture from being selected. The weight w₅ of thevariance of the chroma values x_(i5) is a negative value.

Generally, when photographing an image, a user intentionally places atarget region at the center of the image. Accordingly, x_(i6) may beused so that a region of which the center of gravity (or the center) isplaced around the center of the image can be selected. The weight w₆ ofthe distance x_(i6) is a negative value.

The outer periphery of a region of a landscape picture contains moreuneven portions than that of a circumscribed rectangle. In contrast, thebackground of, for example, a name card, is a rectangle, and, in mostcases, the outer periphery of the rectangle is linear. Accordingly,x_(i7) may be used so that a region having a short peripheral contourcan be selected while preventing a landscape picture from beingselected. The weight w₇ of the peripheral contour x_(i7) is a negativevalue.

FIG. 6 illustrates an example of the processing performed by theinspection-region selecting module 130. The calculation result of thediscriminant function for the region image 510 is 1.5, as expressed byequation (2).

$\begin{matrix}{{f\left( {X_{1};w} \right)} = {\sum\limits_{j = 1.5}\;{w_{j}x_{1j}}}} & (2)\end{matrix}$

The calculation result of the discriminant function for the region image520 is 2.0, as expressed by equation (3).

$\begin{matrix}{{f\left( {X_{2};w} \right)} = {\sum\limits_{j = 2.0}\;{w_{j}x_{2j}}}} & (3)\end{matrix}$

The calculation result of the discriminant function for the region image530 is 0.2, as expressed by equation (4).

$\begin{matrix}{{f\left( {X_{3};w} \right)} = {\sum\limits_{j = 0.2}\;{w_{j}x_{3j}}}} & (4)\end{matrix}$

Then, the maximum of the calculated values of the discriminantfunctions, as expressed by equation (5), is selected.

$\begin{matrix}{X^{*} = {\underset{i}{\arg\;\max}\;{f\left( {X_{i};w} \right)}}} & (5)\end{matrix}$

In this case, the region corresponding to the region image 520 isselected, as expressed by equation (6).X*=X ₂  (6)

In step S208, the pixel-value measuring module 140 scans the lines ofthe region in accordance with a predetermined rule so as to measure thepixel values in the region.

FIGS. 7A through 8B illustrate an example of the processing performed bythe pixel-value measuring module 140.

An extracted region image 700 is an image obtained by performing alogical AND operation between the image 330 received in step S202 andthe region image 520 selected in step S206. That is, the region having awhite background at the right side of the photographed image 330 isextracted. Scanning is performed on the extracted region image 700 inthe vertical and horizontal directions at predetermined intervals (e.g.,at equal intervals). For example, lines 702 through 718 are scanned inthe horizontal direction and lines 722 through 742 are scanned in thevertical direction. Then, a graph is plotted by connecting the luminancevalues of the pixels. An example of a graph representing the luminancevalues of pixels obtained by scanning the line 702 is shown in FIG. 7B.Since the line 702 is not contained in the selected region, the graph isconstituted of only invalid data. An example of the graph obtained byscanning the line 712 is shown in FIG. 7C. Since part of the line 702 iscontained in the selected region, the graph is constituted of valid dataand invalid data. In the graph shown in FIG. 7C, the luminance valuedecreases toward the right side. This is because the fog image becomesdarker, on the whole, toward the upper right side.

A determination regarding whether the graph contains valid data orinvalid data can be made by scanning the region image (mask image, i.e.,the region image 520 shown in FIG. 6) selected by the inspection-regionselecting module 130. Positions within the black portion in the regionimage 520 correspond to positions containing valid data in the extractedregion image 700, while positions within the white portion in the regionimage 520 correspond to positions containing invalid data in theextracted region image 700. In the graphs of this example, the luminancevalue of the invalid data is 0.

As the predetermined rule employed when scanning the extracted regionimage 700, not only the vertical and horizontal scanning shown in FIG.7A, but also scanning shown in FIG. 8A may be performed. Morespecifically, in the scanning shown in FIG. 8A, scanning may beperformed at predetermined intervals in an oblique direction from thetop left side to the bottom right side and an oblique direction from thetop right side to the bottom left side. Alternatively, scanning may beperformed, as shown in FIG. 8B, in an elliptical shape at predeterminedintervals. In this case, examples of the elliptical shape may include acircle. The scanning direction may be selected by an operator inaccordance with the gradient of the luminance values of fog, or may bedetermined in advance in accordance with an image capturing device.

In step S210, the pixel-value distribution estimating module 150estimates the gradient of degree of fog (function representing thedegree of fog) from a set of the measured pixel values. As the functionrepresenting the degree of fog estimated in this example, a linearfunction is used by way of example.

FIGS. 9A through 10C illustrate an example of the processing performedby the pixel-value distribution estimating module 150.

As shown in the example of FIG. 9A, the function representing the degreeof fog in the vertical scanning is represented by equation (7), and thefunction representing the degree of fog in the horizontal direction isrepresented by equation (8).c _(j)(y;y _(j),δ_(j))=γ_(j)+δ_(j) x  (7)r _(j)(x;α _(i),β_(i))=α_(i)+β_(i) x  (8)

In the example shown in FIG. 9B, lines including valid data areindicated by the solid lines, while lines including only invalid dataare indicated by the dotted lines. The lines including valid data arethe lines 704 through 716 and the lines 732 through 740. The linesincluding only invalid data are the lines 702 and 718 and the lines 722through 730 and 742.

FIG. 10A illustrates a graph representing a line including valid data.Each of the lines 704 through 716 can be represented by a graph similarto the graph shown in FIG. 10A. A function represented by each of thegraphs of the luminance values is estimated by using the least squaresmethod, etc. For example, the function represented by a graph of theluminance values obtained by scanning the horizontal lines can beexpressed by equation (9). In each of the lines, if the number of validdata elements is smaller than a predetermined number, the function isestimated without using those valid data elements. In the example shownin FIG. 10B, equation (9) is represented by the one-dot chain line.r _(i)(x;{circumflex over (α)} _(i),{circumflex over (β)}_(i))  (9)

The function is applicable to the lines without valid data. Accordingly,in order to determine luminance values at estimation points 932 through940 on the line 702 without valid data, as shown in FIG. 9C, thefunction is applied, as shown in FIG. 10C, to determine the luminancevalues at the estimation points 932 through 940. This means that theestimated function applies to the entire image.

Estimation of the function using vertical lines is performed similarlyto estimation of the function using horizontal lines.

In step S212, the fog eliminating module 160 calculates the amount offog to be eliminated by using the gradient of degree of fog (functionestimated in step S210), thereby correcting the pixel value of the imageat each set of coordinates of the graph. As a result of the correctionperformed in step S212, fog is eliminated from the photographed image soas to reconstruct the original image.

FIGS. 11 through 13 illustrate an example of the processing performed bythe fog eliminating module 160.

Calculation of the amounts of fog at four points of intersection 1112,1114, 1116, and 1118 within a region 1110 shown in FIG. 11 will bedescribed below with reference to FIG. 12. The point of intersection1112 is the intersection between the lines 706 and 726. The point ofintersection 1114 is the intersection between the lines 706 and 728. Thepoint of intersection 1116 is the intersection between the lines 708 and726. The point of intersection 1118 is the intersection between thelines 708 and 728. Those points of intersection 1112 through 1118 arecontained in a region without valid data.

The luminance value at the point of intersection 1112 is calculated byusing equation (10).

$\begin{matrix}{{p\left( {x_{i},y_{j}} \right)} = \frac{{r_{i}\left( {{x_{i};{\hat{\alpha}}_{i}},{\hat{\beta}}_{i}} \right)} + {c_{j}\left( {{y_{j};{\hat{\gamma}}_{j}},{\hat{\delta}}_{j}} \right)}}{2}} & (10)\end{matrix}$

The luminance value calculated by using equation (10) is the average ofthe function r_(i) of the line 706 and the function c_(j) of the line726 at the point of intersection 1112 (x_(i), y_(j)). Similarly, theluminance value at the point of intersection 1114 is the average of thefunction r_(i) of the line 706 and the function c_(i+1) of the line 728at the point of intersection 1114 (x_(i+1), y_(j)). The luminance valueat the point of intersection 1116 is the average of the function r_(i+1)of the line 708 and the function c_(j) of the line 726 at the point ofintersection 1116 (x_(i), y_(j+1)). The luminance value at the point ofintersection 1118 is the average of the function r_(i+1) of the line 708and the function c_(j+1) of the line 728 at the point of intersection1118 (x_(i+1), y_(j+1)). In this manner, the pixel value of a pixel at apoint of intersection is set as the average value of the function of thecorresponding horizontal line and the function of the correspondingvertical line.

Then, the luminance value of a pixel which is not positioned at a pointof intersection, e.g., the luminance value of a pixel positioned at anintermediate point 1222 shown in FIG. 12, is calculated by usingequation (11).p(x,y)=ΣW((x,y),(x _(i) ,y _(j)))p(x _(i) ,y _(j))  (11)In equation (11), the luminance value of the pixel at the intermediatepoint 1222 is calculated by using the luminance values of the pixels atthe four points of intersection surrounding the intermediate point 1222.Accordingly, calculating the luminance value of a pixel which is notpositioned at a point of intersection is performed after calculating theluminance values of the pixels at the surrounding points ofintersection.

In equation (11), the position of the intermediate point 1222 isrepresented by (x, y). W((x, y), (x_(i), y_(j))) on the right side isthe distance weight. That is, W((x, y), (x_(i), y_(j))) is a weight thatuses the distance from the intermediate point 1222 to each of the fourpoints of intersection 1112, 1114, 1116, and 1118. The total of theweights of the four points of intersection is one, and the weight is thereciprocal of the distance. Accordingly, if the distances from theintermediate point 1222 to the four points of intersection 1112 through1118 are equal to each other, the luminance value at the intermediatepoint 1222 is obtained by adding together ¼ of the luminance values atthe points of intersection 1112 through 1118.

Then, the amount of fog is calculated by using equation (12):Δ(x,y)=p*−p(x,y)  (12)where p* is the luminance value, which serves as a predeterminedcorrection target value, and may be determined as follows. The luminancevalues at plural points of intersection (may be all the points ofintersection in an image or may be the points of intersection in aregion selected by the inspection-region selecting module 130) arecollected so as to generate a luminance histogram. Then, in theluminance histogram, the luminance value at a predetermined points ofintersection (e.g., a point of intersection having a luminance valuelower than the brightest luminance by 10%) may be set as p*.Alternatively, an operator may set a certain luminance value for p*.

FIG. 13 illustrates a process of eliminating fog from an image, which isrepresented three-dimensionally. In FIG. 13, the x axis and the y axisdesignate a position in the image, and the z axis is a spacerepresenting the luminance value. Along the z axis, the gray planepositioned at the lower side is a fog image, which covers the entireimage. The pale hatched plane positioned at the upper side designatesthe corrected luminance value of the image. As shown in FIG. 13, p* isthe bright luminance value determined from the luminance histogram. Theluminance values at the individual points of intersection are increasedto p*, thereby eliminating fog. FIG. 13 shows that the luminance at theintermediate point 1222 is being increased.

In step S214, the output module 170 outputs an image from which fog hasbeen eliminated. For example, a printer prints the image from which foghas been eliminated.

In comparison with this exemplary embodiment, a comparative example willbe described below with reference to FIGS. 14C through 14F. Within thephotographed image 1430 shown in FIG. 14C, a line 1432 is scanned, andthe pixel values obtained as a result of the scanning are represented bya graph shown in FIG. 14D. That is, since the fog image 1420 is darkertoward the right side, the luminance value decreases toward the rightside accordingly. However, the luminance values of the original imageare also reflected in the pixel values shown in FIG. 14D, and thus, theluminance values are progressively changed, as indicated by luminancedistributions 1442, 1444, 1446, and 1448. Then, if the luminance valuesare corrected toward the target value, as shown in FIG. 14F, theresulting image is seen as a corrected image 1440 shown in FIG. 14E. Inthe corrected image 1440, in a portion without a white background, suchas in the region 1412, the luminance becomes too bright so that theoriginal red color has been removed. The reason for this phenomenon isas follows. The target for the correction processing in this comparativeexample is a paper document, which is entirely white. However, thiscorrection processing is uniformly performed on a document containing acolor portion and a white background, despite the fact that theluminance value of a chromatic color portion is lower than that of awhite background.

In contrast, in this exemplary embodiment, the region 1414 is selected,and on the basis of the region 1414, the amount of fog is calculated,thereby correcting the luminance values of the image. As a result, theluminance of a portion without a white background, such as the region1412, does not become too bright in contrast to the corrected image 1440shown in FIG. 14E.

An example of the hardware configuration of the image processingapparatus according to this exemplary embodiment will be described belowwith reference to FIG. 15. The hardware configuration shown in FIG. 15may be constituted of a personal computer (PC), and includes a datareading device 1517, such as a scanner, and a data output device 1518,such as a printer.

A CPU 1501 is a controller that executes processing in accordance with acomputer program in which execution sequences of individual modules,such as the non-edge region extracting module 120, the inspection-regionselecting module 130, the pixel-value measuring module 140, thepixel-value distribution estimating module 150, and the fog eliminatingmodule 160, are described.

A read only memory (ROM) 1502 stores therein programs, computationparameters, etc. used by the CPU 1501. A RAM 1503 stores thereinprograms used by the execution of the CPU 1501 and parameters that areappropriately changed in the execution of the CPU 1501. The CPU 1501,the ROM 1502, and the RAM 1503 are connected to one another via a hostbus 1504, such as a CPU bus.

The host bus 1504 is connected to an external bus 1506, such as aperipheral component interconnect (PCI)/interface bus, via a bridge1505.

A keyboard 1508 and a pointing device 1509, such as a mouse, are inputdevices operated by an operator. A display 1510 includes a liquidcrystal display (LCD) or a cathode ray tube (CRT), and displays variousinformation as text or image information.

A hard disk drive (HDD) 1511 has a built-in hard disk and drives thehard disk to cause it to record or play back programs executed by theCPU 1501 or information. In the hard disk, received images, images ofextracted regions, measured pixel values, estimated functions, etc. arestored. Additionally, various computer programs, such as various dataprocessing programs, are also stored.

A drive 1512 reads data or a program recorded on a removable recordingmedium 1513, such as a magnetic disk, an optical disc, a magneto-opticaldisk, or a semiconductor memory, set in the drive 1512, and supplies theread data or program to the RAM 1503, which is connected to the drive1512 via an interface 1507, the external bus 1506, the bridge 1505, andthe host bus 1504. The removable recording medium 1513 is also usable asa data recording area, which is similar to a hard disk.

A connection port 1514 is a port for connecting an external connectiondevice 1515 to the image processing apparatus, and includes a connectingsection, such as a universal serial bus (USB) or IEEE-1394 port. Theconnection port 1514 is connected to the CPU 1501 via the interface1507, the external bus 1506, the bridge 1505, and the host bus 1504. Acommunication device 1516 is connected to a network, and performs datacommunication processing with external devices. The data reading device1517 is, for example, a scanner, and reads documents. The data outputdevice 1518 is, for example, a printer, and outputs documents.

The hardware configuration shown in FIG. 15 is an example only, and anyconfiguration may be employed for the image processing apparatus as longas it can execute the modules described in this exemplary embodiment.For example, some of the modules may be constituted of dedicatedhardware, such as an application specific integrated circuit (ASIC), ormay be contained in an external system and connected to the imageprocessing apparatus via a communication circuit. Alternatively, two ormore of the systems shown in FIG. 15 may be connected via acommunication circuit and operated in cooperation with one another. Thesystem may be integrated in a digital camera, a copying machine, a faxmachine, a scanner, a printer, a multifunction device (an imageprocessing apparatus including two or more functions of a scanner, aprinter, a copying machine, a fax machine, etc.)

The above-described programs may be stored in a recording medium and beprovided. The program recorded on a recording medium may be provided viaa communication medium. In this case, the above-described program may beimplemented as a “computer readable medium storing the program therein”in an exemplary embodiment of the invention.

The “computer readable medium storing a program therein” is a recordingmedium storing a program therein that can be read by a computer, and isused for installing, executing, and distributing the program.

Examples of the recording medium include digital versatile disks (DVDs),and more specifically, DVDs standardized by the DVD Forum, such asDVD-R, DVD-RW, and DVD-RAM, DVDs standardized by the DVD+RW Alliance,such as DVD+R and DVD+RW, compact discs (CDs), and more specifically, aread only memory (CD-ROM), a CD recordable (CD-R), and a CD rewritable(CD-RW), Blu-ray disc (registered), a magneto-optical disk (MO), aflexible disk (FD), magnetic tape, a hard disk, a ROM, an electricallyerasable programmable read only memory (EEPROM) (registered), a flashmemory, a RAM, etc.

The entirety or part of the above-described program may be recorded onsuch a recording medium and stored therein or distributed.Alternatively, the entirety or part of the program may be transmittedthrough communication by using a transmission medium, such as a wirednetwork used for a local area network (LAN), a metropolitan area network(MAN), a wide area network (WAN), the Internet, an intranet, or anextranet, a wireless communication network, or a combination suchnetworks. The program may be transmitted by using carrier waves.

The above-described program may be part of another program, or may berecorded, together with another program, on a recording medium. Theprogram may be divided and recorded on plural recording media. Further,the program may be recorded in any form, e.g., it may be compressed orencrypted, as long as it can be reconstructed.

The foregoing description of the exemplary embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. An image processing apparatus comprising at leastone processor which implements: a receiver that receives an image; anextractor that extracts, according to a determination of non-edgeregions, regions from the image received by the receiver; a selectorthat selects, in accordance with a predetermined rule, a region fromamong the regions extracted by the extractor; a measurer that measuresluminance values of pixels contained in the region selected by theselector; an estimator that estimates, based on the luminance values ofthe pixels measured by the measurer, a function representing a degree offog in the image received by the receiver; and an eliminator thateliminates, on the basis of the function estimated by the estimator, fogfrom the image received by the receiver, wherein the extractor extractsregions which are equal to or greater than a predetermined size orregions which are greater than the predetermined size, and in responseto such regions not being found, processing using the selector,processing using the measurer, processing using the estimator, andprocessing using the eliminator are not performed.
 2. The imageprocessing apparatus according to claim 1, wherein the predeterminedrule employed in the selector is employed based on a size of each of theregions extracted by the extractor.
 3. The image processing apparatusaccording to claim 2, wherein the predetermined rule employed in theselector is employed based also on at least one of luminance values andchroma values of pixels contained in each of the regions extracted bythe extractor.
 4. The image processing apparatus according to claim 2,wherein the predetermined rule employed in the selector is employedbased also on at least one of a variance of chroma values of pixelscontained in each of the regions extracted by the extractor, a positionof each of the regions in the image, and a size of an outer periphery ofeach of the regions.
 5. The image processing apparatus according toclaim 1, wherein the measurer measures the luminance values of thepixels by scanning the region selected by the selector in at least oneof a horizontal direction, a vertical direction, an oblique direction,and a direction in an elliptical shape.
 6. The image processingapparatus according to claim 1, wherein the estimator estimates thefunction based on a connecting of the luminance values of the pixelscontained in the region selected by the selector.
 7. The imageprocessing apparatus according to claim 6, wherein, in response to themeasurer scanning the region selected by the selector in a plurality ofscanning directions in order to measure luminance values of pixelscontained in the region selected by the selector, the eliminatorcalculates a value of fog at a position at which the plurality ofscanning directions intersect with each other on the basis of aplurality of functions, which correspond to the plurality of scanningdirections, estimated by the estimator.
 8. An image processing methodcomprising: receiving an image; extracting, according to a determinationof non-edge regions, regions from the received image; selecting, inaccordance with a predetermined rule, a region from among the extractedregions; measuring luminance values of pixels contained in the selectedregion; estimating, based on the measured luminance values of thepixels, a function representing a degree of fog in the received image;and eliminating, on the basis of the estimated function, fog from thereceived image, wherein the extracting comprises extracting regionswhich are equal to or greater than a predetermined size or regions whichare greater than the predetermined size, and in response to such regionsnot being found, the selecting, the measuring, the estimating, and theeliminating are not performed.
 9. A non-transitory computer readablemedium storing a program causing a computer to execute a process, theprocess comprising: receiving an image; extracting, according to adetermination of non-edge regions, regions from the received image;selecting, in accordance with a predetermined rule, a region from amongthe extracted regions; measuring luminance values of pixels contained inthe selected region; estimating, based on the measured luminance valuesof the pixels, a function representing a degree of fog in the receivedimage; and eliminating, on the basis of the estimated function, fog fromthe received image, wherein the extracting comprises extracting regionswhich are equal to or greater than a predetermined size or regions whichare greater than the predetermined size, and in response to such regionsnot being found, the selecting, the measuring, the estimating, and theeliminating are not performed.
 10. The image processing apparatusaccording to claim 1, wherein the extractor detects edges of verticallines or horizontal lines in the received image, performs a binarizingprocessing on the image to turn the detected edges to a first color andremaining portions to a second color, and performs labeling processingon the binarizing processed image so as to attach a same label toregions of continuous pixels of the second color.
 11. The imageprocessing apparatus according to claim 10, wherein the extractoreliminates noise from the labeling processed image by eliminatingregions smaller than a predetermined size.
 12. The image processingapparatus according to claim 1, wherein the selector determines, inaccordance with the predetermined rule, a background region as theregion from among the regions extracted by the extractor.
 13. The imageprocessing apparatus according to claim 1, wherein, in response to themeasurer scanning the region selected by the selector in a plurality ofscanning directions in order to measure luminance values of pixelscontained in the region selected by the selector, the eliminatorcalculates a value of fog at a position of a first pixel which is not onany scanning line by using, in accordance with a distance between thefirst pixel and the second pixel, a value of fog at a position of asecond pixel which is on a scanning line.
 14. The image processingapparatus according to claim 1, wherein the eliminator eliminates, onthe basis of the function estimated by the estimator, the fog bycalculating a value of fog which corresponds to a difference between apixel value of the received image and a corresponding pixel value of anoriginal image.
 15. An image processing apparatus comprising at leastone processor which implements: a storage that stores image informationof image having a fog, a subject of the image having a chromatic colorregion; an extractor that extracts, according to a determination ofnon-edge regions, regions from the image stored in the storage; aselector that selects, in accordance with a predetermined rule, a regionfrom among the regions extracted by the extractor; a measurer thatmeasures luminance values of pixels contained in the region selected bythe selector, each pixel having coordinates in the image; an estimatorthat estimates, based on the luminance value of the pixels measured bythe measurer, a function representing a degree of fog for each of thecoordinates of the pixels in the image stored in the storage; and aneliminator that eliminates, on the basis of the function estimated bythe estimator, fog from the image stored in the storage, wherein theextractor extracts regions which are equal to or greater than apredetermined size or regions which are greater than the predeterminedsize, and in response to such regions not being found, processing usingthe selector, processing using the measurer, processing using theestimator, and processing using the eliminator are not performed.